Optimized cache allocation algorithm for multiple speculative requests

ABSTRACT

A method of operating a computer system is disclosed in which an instruction having an explicit prefetch request is issued directly from an instruction sequence unit to a prefetch unit of a processing unit. In a preferred embodiment, two prefetch units are used, the first prefetch unit being hardware independent and dynamically monitoring one or more active streams associated with operations carried out by a core of the processing unit, and the second prefetch unit being aware of the lower level storage subsystem and sending with the prefetch request an indication that a prefetch value is to be loaded into a lower level cache of the processing unit. The invention may advantageously associate each prefetch request with a stream ID of an associated processor stream, or a processor ID of the requesting processing unit (the latter feature is particularly useful for caches which are shared by a processing unit cluster). If another prefetch value is requested from the memory hiearchy and it is determined that a prefetch limit of cache usage has been met by the cache, then a cache line in the cache containing one of the earlier prefetch values is allocated for receiving the other prefetch value.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present invention is related to the following applications filedconcurrently with this application: U.S. patent application Ser. No.09/345,642 entitled “METHOD FOR INSTRUCTION EXTENSIONS FOR A TIGHTLYCOUPLED SPECULATIVE REQUEST UNIT”; U.S. patent application Ser. No.09/345,643 entitled “LAYERED SPECULATIVE REQUEST UNIT WITH INSTRUCTIONOPTIMIZED AND STORAGE HIERARCHY OPTIMIZED PARTITIONS”; U.S. patentapplication Ser. No. 09/345,644 entitled “EXTENDED CACHE STATE WITHPREFETCHED STREAM ID INFORMATION”; U.S. patent application Ser. No.09/345,713 entitled “CACHE ALLOCATION POLICY BASED ON SPECULATIVEREQUEST HISTORY”; U.S. patent application Ser. No. 09/345,715 entitled“MECHANISM FOR HIGH PERFORMANCE TRANSFER OF SPECULATIVE REQUEST DATABETWEEN LEVELS OF CACHE HIERARCHY”; U.S. patent application Ser. No.09/345,716 entitled “TIME BASED MECHANISM FOR CACHED SPECULATIVE DATADEALLOCATION”.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to computer systems, and morespecifically to an improved method of prefetching values (instructionsor operand data) used by a processor core of a computer system. Inparticular, the present invention makes more efficient use of a cachehierarchy working in conjunction with prefetching (speculativerequests).

2. Description of Related Art

The basic structure of a conventional computer system includes one ormore processing units connected to various input/output devices for theuser interface (such as a display monitor, keyboard and graphicalpointing device), a permanent memory device (such as a hard disk, or afloppy diskette) for storing the computer's operating system and userprograms, and a temporary memory device (such as random access memory orRAM) that is used by the processor(s) in carrying out programinstructions. The evolution of computer processor architectures hastransitioned from the now widely-accepted reduced instruction setcomputing (RISC) configurations, to so-called superscalar computerarchitectures, wherein multiple and concurrently operable executionunits within the processor are integrated through a plurality ofregisters and control mechanisms.

The objective of superscalar architecture is to employ parallelism tomaximize or substantially increase the number of program instructions(or “micro-operations”) simultaneously processed by the multipleexecution units during each interval of time (processor cycle), whileensuring that the order of instruction execution as defined by theprogrammer is reflected in the output. For example, the controlmechanism must manage dependencies among the data being concurrentlyprocessed by the multiple execution units, and the control mechanismmust ensure that integrity of sequentiality is maintained in thepresence of precise interrupts and restarts. The control mechanismpreferably provides instruction deletion capability such as is neededwith instruction-defined branching operations, yet retains the overallorder of the program execution. It is desirable to satisfy theseobjectives consistent with the further commercial objectives ofminimizing electronic device count and complexity.

An illustrative embodiment of a conventional processing unit forprocessing information is shown in FIG. 1, which depicts thearchitecture for a PowerPC™ microprocessor 12 manufactured byInternational Business Machines Corp. (IBM—assignee of the presentinvention). Processor 12 operates according to reduced instruction setcomputing (RISC) techniques, and is a single integrated circuitsuperscalar microprocessor. As discussed further below, processor 12includes various execution units, registers, buffers, memories, andother functional units, which are all formed by integrated circuitry.

Processor 12 is coupled to a system bus 20 via a bus interface unit(BIU) 30 within processor 12. BIU 30 controls the transfer ofinformation between processor 12 and other devices coupled to system bus20 such as a main memory 18. Processor 12, system bus 20, and the otherdevices coupled to system bus 20 together form a host data processingsystem. Bus 20, as well as various other connections described, includemore than one line or wire, e.g., the bus could be a 32-bit bus. BIU 30is connected to a high speed instruction cache 32 and a high speed datacache 34. A lower level (L2) cache (not shown) may be provided as anintermediary between processor 12 and system bus 20. An L2 cache canstore a much larger amount of information (instructions and operanddata) than the on-board caches can, but at a longer access penalty. Forexample, the L2 cache may be a chip having a storage capacity of 512kilobytes, while the processor may be an IBM PowerPC™ 604-seriesprocessor having on-board caches with 64 kilobytes of total storage. Agiven cache line usually has several memory words, e.g., a 64-byte linecontains eight 8-byte words.

The output of instruction cache 32 is connected to a sequencer unit 36(instruction dispatch unit, also referred to as an instruction sequenceunit or ISU). In response to the particular instructions received frominstruction cache 32, sequencer unit 36 outputs instructions to otherexecution circuitry of processor 12, including six execution units,namely, a branch unit 38, a fixed-point unit A (FXUA) 40, a fixed-pointunit B (FXUB) 42, a complex fixed-point unit (CFXU) 44, a load/storeunit (LSU) 46, and a floating-point unit (FPU) 48.

The inputs of FXUA 40, FXUB 42, CFXU 44 and LSU 46 also receive sourceoperand information from general-purpose registers (GPRs) 50 andfixed-point rename buffers 52. The outputs of FXUA 40, FXUB 42, CFXU 44and LSU 46 send destination operand information for storage at selectedentries in fixed-point rename buffers 52. CFXU 44 further has an inputand an output connected to special-purpose registers (SPRs) 54 forreceiving and sending source operand information and destination operandinformation, respectively. An input of FPU 48 receives source operandinformation from floating-point registers (FPRs) 56 and floating-pointrename buffers 58. The output of FPU 48 sends destination operandinformation to selected entries in floating-point rename buffers 58.

As is well known by those skilled in the art, each of execution units38-48 executes one or more instructions within a particular class ofsequential instructions during each processor cycle. For example, FXUA42 performs fixed-point mathematical operations such as addition,subtraction, ANDing, ORing, and XORing utilizing source operandsreceived from specified GPRs 50. Conversely, FPU 48 performsfloating-point operations, such as floating-point multiplication anddivision, on source operands received from FPRs 56. As its name implies,LSU 46 executes floating-point and fixed-point instructions which eitherload operand data from memory (i.e., from data cache 34) into selectedGPRs 50 or FPRs 56, or which store data from selected GPRs 50 or FPRs 56to memory 18. Processor 12 may include other registers, such asconfiguration registers, memory management registers, exception handlingregisters, and miscellaneous registers, which are not shown.

Processor 12 carries out program instructions from a user application orthe operating system, by routing the instructions and operand data tothe appropriate execution units, buffers and registers, and by sendingthe resulting output to the system memory device (RAM), or to someoutput device such as a display console or printer. A computer programcan be broken down into a collection of processes which are executed bythe processor(s). The smallest unit of operation to be performed withina process is referred to as a thread. The use of threads in modernoperating systems is well known. Threads allow multiple execution pathswithin a single address space (the process context) to run concurrentlyon a processor. This “multithreading” increases throughput in amulti-processor system, and provides modularity in a uniprocessorsystem.

One problem with conventional processing is that operations are oftendelayed as they must wait on an instruction or item of data beforeprocessing of a thread may continue. One way to mitigate this effect iswith multithreading, which allows the processor to switch its contextand run another thread that is not dependent upon the requested value.Another approach to reducing overall memory latency is the use ofcaches, as discussed above. A related approach involves the prefetchingof values. “Prefetching” refers to the speculative retrieval of values(operand data or instructions) from the memory hierarchy, and thetemporary storage of the values in registers or buffers near theprocessor core, before they are actually needed. Then, when the value isneeded, it can quickly be supplied to the sequencer unit, after which itcan be executed (if it is an instruction) or acted upon (if it is data).Prefetch buffers differ from a cache in that a cache may contain valuesthat were loaded in response to the actual execution of an operation (aload or i-fetch operation), while prefetching retrieves values prior tothe execution of any such operation.

An instruction prefetch queue may hold, e.g., eight instructions toprovide look-ahead capability. Branch unit 38 searches the instructionqueue in sequencer unit 36 (typically only the bottom half of the queue)for a branch instruction and uses static branch prediction on unresolvedconditional branches to allow the IFU to speculatively requestinstructions from a predicted target instruction stream while aconditional branch is evaluated (branch unit 38 also folds out branchinstructions for unconditional branches). Static branch prediction is amechanism by which software (for example, a compiler program) can give ahint to the computer hardware about the direction that the branch islikely to take. In this manner, when a correctly predicted branch isresolved, instruction execution continues without interruption along thepredicated path. If branch prediction is incorrect, the IFU flushes allinstructions from the instruction queue. Instruction issue then resumeswith the instruction from the correct path.

A prefetch mechanism for operand data may also be provided within businterface unit 30. This prefetch mechanism monitors the cache operations(i.e., cache misses) and detects data streams (requests to sequentialmemory lines). Based on the detected streams and using known patterns,BIU 30 speculatively issues requests for operand data which have not yetbeen requested. BIU 30 can typically have up to four outstanding(detected) streams. Reload buffers are used to store the data untilrequested by data cache 34.

In spite of such approaches to reducing the effects of memory latencies,there are still significant delays associated with operations requiringmemory access. As alluded to above, one cause of such delays is theincorrect prediction of a branch (for instructions) or a stream (foroperand data). In the former case, the unused, speculatively requestedinstructions must be flushed, directly stalling the core. In the lattercase, missed data is not available in the prefetch reload queues, and aconsiderable delay is incurred while the data is retrieved fromelsewhere in the memory hierarchy. Much improvement is needed in theprefetching mechanism.

Another cause of significant delay is related to the effects thatprefetching has on the cache hierarchy. For example, in multi-levelcache hierarchies, it might be efficient under certain conditions toload prefetch values into lower cache levels, but not into upper cachelevels. Also, when a speculative prefetch request misses a cache, therequest may have to be retried an excessive number of times (when thelower level storage subsystem is busy), which unnecessarily wastes busbandwidth, and the requested value might not ever be used. Furthermore,a cache can easily become “polluted” with speculative request data,i.e., the cache contains so much prefetch data that demand requests(those requests arising from actual load or i-fetch operations)frequently miss the cache. In this case the prefetch mechanism hasoverburdened the capacity of the cache, which can lead to thrashing. Thecache replacement/victimization algorithm (such as a least-recentlyused, or LRU, algorithm) cannot account for the nature of the prefetchrequest. Moreover, after prefetched data has been used by the core (andis no longer required), it may stay in the cache for a relatively longtime due to the LRU algorithm and might thus indirectly contribute tofurther cache misses (which is again particularly troublesome withmisses of demand requests, rather than speculative requests). Finally,in multi-processor systems wherein one or more caches are shared by aplurality of processors, prefetching can result in uneven (andinefficient) use of the cache with respect to the sharing processors.

In light of the foregoing, it would be desirable to provide a method ofspeeding up core processing by improving the prefetching mechanism,particularly with respect to its interactions with the cache hierarchy.It would be further advantageous if the method allowed a programmer tooptimize various features of the prefetching mechanism.

SUMMARY OF THE INVENTION

It is therefore one object of the present invention to provide animproved processor for a computer system, having a prefetch mechanismfor instructions and/or operand data.

It is another object of the present invention to provide an improveddata processing system using such a processor, which also has one ormore caches in the memory hierarchy.

It is yet another object of the present invention to provide a computersystem which makes more efficient use of a cache hierarchy working inconjunction with prefetching.

The foregoing objects are achieved in a method of accessing a memoryhierarchy of a computer system, comprising the steps of loading at leastone prefetch value into a cache of the memory hierarchy, requesting anew value from the memory hierarchy, determining that a prefetch limitof cache usage has been met by the cache and, in response to saiddetermining step, allocating a cache line in the cache containing aprefetch value, for receiving the new value. The speculatively requestedvalue can be either operand data or an instruction. The prefetch limitof cache usage can be established using a maximum number of sets in aparticular congruence class of the cache. In the preferredimplementation, when the prefetch value is loaded, a bit is set in thecache to indicate that a cache line is being loaded with a prefetchvalue, and a congruence class set number of the cache line is loadedinto a prefetch set ID field.

The above as well as additional objectives, features, and advantages ofthe present invention will become apparent in the following detailedwritten description.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features believed characteristic of the invention are setforth in the appended claims. The invention itself, however, as well asa preferred mode of use, further objectives, and advantages thereof,will best be understood by reference to the following detaileddescription of an illustrative embodiment when read in conjunction withthe accompanying drawings, wherein:

FIG. 1 is a block diagram of a conventional superscalar computerprocessor, depicting execution units, buffers, registers, and theon-board (L1) data and instruction caches;

FIG. 2 is an illustration of one embodiment of a data processing systemin which the present invention can be practiced;

FIG. 3 is a block diagram illustrating selected components that can beincluded in the data processing system of FIG. 2 according to theteachings of the present invention;

FIG. 4 is a block diagram of a central processing unit constructed inaccordance with the present invention, having a multi-level cachehierarchy, and a prefetch unit linked to an instruction sequence unitwhich provides instructions to other execution units of the processorcore;

FIG. 5 is a block diagram of the prefetch unit of FIG. 4 according toone implementation of the present invention;

FIG. 6 is a block diagram of a lower level (e.g., L2) cache having adirectory, an entry array, and a least-recently used (LRU) victimselection unit, wherein certain flags or bits are provided in thedirectory records and the LRU unit in accordance with a furtherimplementation of the present invention; and

FIG. 7 is a block diagram of a multi-processor, multi-level cache dataprocessing system constructed in accordance with the present inventionwherein certain caches are shared by more than one processor core.

DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT

With reference now to the figures, and in particular with reference toFIG. 2, a data processing system 120 is shown in which the presentinvention can be practiced. The data processing system 120 includesprocessor 122, keyboard 182, and display 196. Keyboard 182 is coupled toprocessor 122 by a cable 128. Display 196 includes display screen 130,which may be implemented using a cathode ray tube (CRT), a liquidcrystal display (LCD), an electrode luminescent panel or the like. Thedata processing system 120 also includes pointing device 184, which maybe implemented using a track ball, a joy stick, touch sensitive tabletor screen, track path, or as illustrated a mouse. The pointing device184 may be used to move a pointer or cursor on display screen 130.Processor 122 may also be coupled to one or more peripheral devices sucha modem 192, CD-ROM 178, network adapter 190, and floppy disk drive 140,each of which may be internal or external to the enclosure or processor122. An output device such as a printer 100 may also be coupled withprocessor 122.

It should be noted and recognized by those persons of ordinary skill inthe art that display 196, keyboard 182, and pointing device 184 may eachbe implemented using any one of several known off-the-shelf components.

Reference now being made to FIG. 3, a high level block diagram is shownillustrating selected components that can be included in the dataprocessing system 120 of FIG. 2 according to the teachings of thepresent invention. The data processing system 120 is controlledprimarily by computer readable instructions, which can be in the form ofsoftware, wherever, or by whatever means such software is stored oraccessed. Such software may be executed within the Central ProcessingUnit (CPU) 150 to cause data processing system 120 to do work.

Memory devices coupled to system bus 105 include Random Access Memory(RAM) 156, Read Only Memory (ROM) 158, and nonvolatile memory 160. Suchmemories include circuitry that allows information to be stored andretrieved. ROMs contain stored data that cannot be modified. Data storedin RAM can be changed by CPU 150 or other hardware devices. Nonvolatilememory is memory that does not lose data when power is removed from it.Nonvolatile memories include ROM, EPROM, flash memory, or battery-packCMOS RAM. As shown in FIG. 3, such battery-pack CMOS RAM may be used tostore configuration information.

An expansion card or board is a circuit board that includes chips andother electronic components connected that adds functions or resourcesto the computer. Typically, expansion cards add memory, disk-drivecontrollers 166, video support, parallel and serial ports, and internalmodems. For lap top, palm top, and other portable computers, expansioncards usually take the form of PC cards, which are credit card-sizeddevices designed to plug into a slot in the side or back of a computer.An example of such a slot is PCMCIA slot (Personal Computer Memory CardInternational Association) which defines type I, II and III card slots.Thus, empty slots 168 may be used to receive various types of expansioncards or PCMCIA cards.

Disk controller 166 and diskette controller 170 both include specialpurpose integrated circuits and associated circuitry that direct andcontrol reading from and writing to hard disk drive 172, and a floppydisk or diskette 74, respectively. Such disk controllers handle taskssuch as positioning read/write head, mediating between the drive and theCPU 150, and controlling the transfer of information to and from memory.A single disk controller may be able to control more than one diskdrive.

CD-ROM controller 176 may be included in data processing 120 for readingdata from CD-ROM 178 (compact disk read only memory). Such CD-ROMs uselaser optics rather than magnetic means for reading data.

Keyboard mouse controller 180 is provided in data processing system 120for interfacing with keyboard 182 and pointing device 184. Such pointingdevices are typically used to control an on-screen element, such as agraphical pointer or cursor, which may take the form of an arrow havinga hot spot that specifies the location of the pointer when the userpresses a mouse button. Other pointing devices include a graphicstablet, stylus, light pin, joystick, puck, track ball, track pad, andthe pointing device sold under the trademark “Track Point” byInternational Business Machines Corp. (IBM).

Communication between processing system 120 and other data processingsystems may be facilitated by serial controller 188 and network adapter190, both of which are coupled to system bus 105. Serial controller 188is used to transmit information between computers, or between a computerand peripheral devices, one bit at a time over a single line. Serialcommunications can be synchronous (controlled by some standard such as aclock) or asynchronous (managed by the exchange of control signals thatgovern the flow of information). Examples of serial communicationstandards include RS-232 interface and the RS-422 interface. Asillustrated, such a serial interface may be used to communicate withmodem 192. A modem is a communication device that enables a computer totransmit information over standard telephone lines. Modems convertdigital computer signals to interlock signals suitable forcommunications over telephone lines. Modem 192 can be utilized toconnect data processing system 120 to an on-line information service oran Internet service provider. Such service providers may offer softwarethat can be down loaded into data processing system 120 via modem 192.Modem 192 may provide a connection to other sources of software, such asa server, an electronic bulletin board (BBS), or the Internet (includingthe World Wide Web).

Network adapter 190 may be used to connect data processing system 120 toa local area network 194. Network 194 may provide computer users withmeans of communicating and transferring software and informationelectronically. Additionally, network 194 may provide distributedprocessing, which involves several computers in the sharing of workloadsor cooperative efforts in performing a task. Network 194 can alsoprovide a connection to other systems like those mentioned above (a BBS,the Internet, etc.).

Display 196, which is controlled by display controller 198, is used todisplay visual output generated by data processing system 120. Suchvisual output may include text, graphics, animated graphics, and video.Display 196 may be implemented with CRT-based video display, anLCD-based flat panel display, or a gas plasma-based flat-panel display.Display controller 198 includes electronic components required togenerate a video signal that is sent to display 196.

Printer 100 may be coupled to data processing system 120 via parallelcontroller 102. Printer 100 is used to put text or a computer-generatedimage (or combinations thereof) on paper or on another medium, such as atransparency sheet. Other types of printers may include an image setter,a plotter, or a film recorder.

Parallel controller 102 is used to send multiple data and control bitssimultaneously over wires connected between system bus 105 and anotherparallel communication device, such as a printer 100.

CPU 150 fetches, decodes, and executes instructions, and transfersinformation to and from other resources via the computers maindata-transfer path, system bus 105. Such a bus connects the componentsin a data processing system 120 and defines the medium for dataexchange. System bus 105 connects together and allows for the exchangeof data between memory units 156, 158, and 160, CPU 150, and otherdevices as shown in FIG. 3. Those skilled in the art will appreciatethat a data processing system constructed in accordance with the presentinvention may have multiple components selected from the foregoing,including even multiple processors.

Referring now to FIG. 4, one embodiment of the present invention allowsdata processing system 120 to more efficiently process information, byutilizing hints in the instruction set architecture used by theprocessor core of CPU 150 to exploit prefetching. The processor core 200uses several conventional elements, including a plurality of registers,such as general purpose and special purpose registers (not shown), and aplurality of execution units, among them a floating point unit (FPU)202, a fixed point unit (FXU) 204, and any other execution units desiredsuch as a complex fixed-point unit (not shown). FPU 202 performsfloating-point operations, such as floating-point multiplication anddivision, on source operands received from floating point registers. FXU204 performs fixed-point mathematical operations such as addition,subtraction, ANDing, ORing, and XORing utilizing source operandsreceived from specified general purpose registers.

Processor core 200 is further comprised of several novel elements suchas an instruction sequence unit (ISU) 206, an instruction fetch unit(IFU) 208, a load/store unit (LSU) 210, and a prefetch unit (PFU) 212.ISU 206, IFU 208 and LSU 210 perform functions which include thoseperformed by conventional execution units, but are further modified toenable the features described hereinafter. IFU 208 executes instructionfetches, while LSU 210 executes floating-point and fixed-pointinstructions which either load operand data from memory, or which storedata to memory. ISU 206 outputs instructions to other executioncircuitry of processor core 200.

IFU 208 and LSU 210 are connected to the on-board (L1) cache. Althoughnot shown in FIG. 4, the L1 cache may actually comprise separate operanddata and instruction caches. L1 cache 214 is further connected to thelower level storage subsystem which, in the illustrated embodiment,includes at least one additional cache level, L2 cache 216. L2 cache 216may in turn be connected to another cache level, or to the main memory(RAM 156), via system bus 105.

PFU 212 is linked to ISU 206. The instruction set architecture (ISA) forthe processor core (e.g., the ISA of a PowerPC™ 630 processor) isextended to include explicit prefetch instructions (speculativerequests). ISU 206 is aware of PFU 212 and issues instructions directlyto PFU according to bits in the extended instruction which are set bythe software (the computer's operating system or user programs). Thisapproach allows the software to better optimize scheduling of load andstore operations (prediction techniques in software may be more accuratethan hardware). PFU 212 may be split into an instruction prefetch unitand an operand data prefetch unit if desired.

With further reference to FIG. 5, in the illustrative embodiment PFU 212is actually comprised of two essentially separate prefetch units 212Aand 212B. Each of these prefetch units may have its own arithmetic logicunit (ALU) functions, its own cache (with a directory, entry array, andreplacement mechanism), and its own translation lookaside buffer (TLB).Prefetch unit A (PFUA) 212A is designed to be hardware independent, anddynamically monitors active streams. Prefetch unit B (PFUB) 212B ishardware dependent, as it is configured to have knowledge of theunderlying storage hierarchy. More specifically, PFUB 212B is adaptedfor the prefetching of values into a lower level cache such as L2 cache216. PFUB 212B is not only aware of the downstream hierarchy in terms ofthe number of cache levels, but may also be aware of theirassociativity, size, and latencies between levels. This information canbe used to determine when it would be more efficient to load the valuesinto a lower level cache without immediately loading them into thehigher level (upstream) cache, the prefetch cache, or other units in thecore. PFUB 212B may thus be used to prefetch values into any lower levelcache, e.g., L3 or L4 (or both the L3 and L4 but not the L2). Theexplicit information in the extended instruction bits can apply to bothPFUA 212A and PFUB 212B.

For set associative caches, the prefetch mechanism of the presentinvention may also be used to indicate which set of the cache line thevalue is to be loaded into. This capability can be used to minimizecache pollution and avoid thrashing and striding which arises fromprefetches. The set can be passed down to the cache, along with otherinformation such as an identification number (ID) that is uniquelyassociated with a particular stream detected by the PFU (PFU 212monitors requests to IFU 208 and LSU 210). The stream ID can be used asdescribed further below. The lower level caches can be designed toabandon speculative prefetches if the request is repeatedly retried bythe downstream storage subsystem. This feature can also be programmablyset by passing another bit from the PFU to the lower level cache.

Referring now to FIG. 6, one implementation of the present inventionuses an L2 cache 216 which provides one or more special flags, for eachdirectory entry, which relate to prefetching. A given entry in thedirectory 218 includes the conventional tag portion, and at least afirst flag 220 which indicates whether the entry was retrieved as theresult of a prefetch operation. This flag can then be scanned duringlater victim selection to allow future prefetches to allocate theprevious prefetched request. Allocating a line containing a previouslyprefetched value, for use by a later speculative request, limits cachepollution by too many prefetched lines.

As also shown in FIG. 6, a directory entry may be provided with a secondflag 222 which is used to indicate whether the speculatively requestedvalue has also been sourced to the next higher cache level. In otherwords, for L2 cache 216, if the second flag 222 is set, this means thatthe prefetch value has already been forwarded to L1 cache 214. Wheneverthe L2 cache misses a congruence class having a line with the secondflag set, that line can automatically be invalidated and allocated forthe later requested value. This feature is particularly useful with anon-inclusive cache, since future L2 misses may allocate the prefetchedline that has already been forwarded to the next higher level, withoutthe need of invalidating the line in the higher level, makingparticularly efficient use of the multi-level cache structure. If thefirst flag 220 is set but second flag 222 is not, meaning that the valuewas prefetched but has not yet been forwarded, then a victim can beselected using a standard victim selection algorithm, such as aleast-recently, or less-recently, used (LRU) algorithm, which is appliedto all sets including the prefetched line. Both of the flags 220 and 222can be used for either instructions or operand data loaded in the cache,and the corresponding features can be made programmable using, e.g.,mode bits.

FIG. 6 also depicts how the present invention allows the LRU unit 224 touse prefetch information in carrying out the victim selection process.In order to avoid filling up the cache with speculative requests, alimit may be designed into LRU unit 224, e.g., only two sets out ofeight to be used for prefetch requests (the number of sets could ofcourse be lower or higher). Two bits 226 in a given congruence class areused to indicate whether any allocated lines in that class are fromprefetch requests. These lines are referred to as prefetch (PF) slots 1and 2. When a prefetch request misses the L2 cache and neither slot isallocated, a victim is selected using the standard LRU algorithm. Bit226 for slot 1 is then flagged, and the number of the set chosen forvictimization is loaded into a set ID field 227 for slot 1. When a laterprefetch request again misses the L2 cache, bit 226 for slot 2 isflagged, and the number of the set chosen for the second prefetchrequest is loaded into the set ID field 227 for slot 2.

The prefetch slots are not exclusively used for speculative requests;they may be used for non-prefetch requests if the standard LRU algorithmwere to selected that set for victimization.

Additionally, a plurality of bits 228 are used to hold the stream ID ofthe stream associated with each prefetch value. In this manner, if afuture prefetch request misses the L2 cache, and its stream ID matchesthat for one of the PF allocated slots 1 or 2, then the matching slot isautomatically selected for victimization (the odds are that, if theoriginal prefetched data is not used by the time another request in thesame stream hits that congruence class, then the original data will notbe used, since sequential memory lines generally are not directed to thesame congruence class). The selection of the matching slot can furtherlimit pollution of the cache with prefetch data. An additional bit 229may optionally be utilized to indicate, as between the two slots, whichis the most recently used, for those cases wherein a prefetch requestmisses the L2 cache and both slots already have prefetched lines withdifferent stream IDs.

If a demand request hits L2 cache 216 on a prefetch value, the LRUalgorithm is updated as normal, but load hits do not affect bit 229. Asa further enhancement, load hits could be used to reset bits 226.

Victimization of a prefetched line may also be based on a time delay,i.e., the amount of time passing since the speculative request wasissued. In those applications wherein the time window for expecteddemand of the prefetch value is well understood, there is no need towait for another cache miss; the line can be immediately invalidatedwhen a timer runs out. The time delay may be fixed or programmable, anda mode bit may be used to turn off the feature altogether. This featurecan effectively be used in conjunction with the flags 220 and 222, bysimply setting flag 222 in response to the timeout, which would have thesame effect of invalidating that line.

The present invention further contemplates improved prefetching within acache hierarchy having vertical caches that support a processor corecluster, as illustrated in FIG. 7. In that implementation, a givenprocessor 200 a, 200 b, 200 c or 200 d has its own on-board (L1) cache,but each L2 cache 216 a, 216 b, supports (is shared by) two processors.Further, an L3 cache 230 supports more than one L2 cache. For such acache hierarchy, the prefetch request may send down (in addition to theinformation previously described) a processor or CPU number. This CPUnumber can then be used by an LRU unit in any of the L2 or L3 caches tofurther optimize the replacement algorithm. For example, the number ofsets in a given congruence class usable by different cores can belimited (e.g., either one of the processors 200 a or 200 b can utilizeat most four sets within a given congruence class in L2 cache 216 a, andat most two sets within a given congruence class in L3 cache 230). Aswith the PF slots described above, each LRU unit can also maintainseparate information for determining the least-recently used line amongthe sets usable by a given core.

Although the invention has been described with reference to specificembodiments, this description is not meant to be construed in a limitingsense. Various modifications of the disclosed embodiments, as well asalternative embodiments of the invention, will become apparent topersons skilled in the art upon reference to the description of theinvention. It is therefore contemplated that such modifications can bemade without departing from the spirit or scope of the present inventionas defined in the appended claims.

What is claimed is:
 1. A method of controlling a cache of a computersystem, said cache including a directory, said method comprising:loading at least one prefetch value and one non-prefetch value intorespective cache blocks of a common congruence class of the cache andmarking a directory entry associated with the cache block containing theprefetch value to indicate that the cache block contains a prefetchvalue; requesting a new value that maps to the congruence class from thememory hierarchy; determining by reference to the directory that aprefetch limit of cache usage has been met by the cache; and in responseto said determining step, preferentially allocating the cache blockcontaining the prefetch value to receive the new value.
 2. The method ofclaim 1, wherein said determining step includes determining that amaximum number of cache blocks in the congruence class currently containprefetch values.
 3. The method of claim 1 wherein said loading steploading an identifying number of the cache block into a prefetch IDfield.
 4. The method of claim 1 wherein said requesting step isperformed in response to the further step of issuing a demand operationfrom an execution unit of the computer system.
 5. The method of claim 1wherein said requesting step is performed in response to the furtherstep of issuing a prefetch operation from a prefetch unit of thecomputer system.
 6. The method of claim 5 wherein said loading steploads a plurality of prefetch values into the cache.
 7. The method ofclaim 6 wherein said allocating step includes the step of selecting acache block within the congruence class for eviction from among onlythose cache blocks that contain prefetch values.
 8. The method of claim6 wherein said selecting step includes the step of preferentiallyselecting for eviction a cache block containing a prefetch value thathas been sourced to a next higher level cache.
 9. The method of claim 1wherein the new value is an instruction, and said allocating stepallocates a cache line in an instruction cache.
 10. The method of claim1 wherein the new value is operand data, and said allocating stepallocates a cache line in a data cache.
 11. A memory hierarchy,comprising: a cache array including a congruence class including aplurality of cache blocks containing at least one non-prefetch value andat least one prefetch value; a directory of contents of the cache array,said directory including an entry associated with a cache blockcontaining said prefetch value; and a cache controller that, responsiveto loading the prefetch value, marks the directory entry to indicatethat cache block contains a prefetch value and, responsive to a requestfor a new value that maps to the congruence class, determines byreference to the directory that a prefetch limit of cache usage has beenmet by the cache and, responsive thereto, preferentially allocates thecache block containing the prefetch value to receive the new value. 12.The memory hierarchy of claim 11, wherein said cache controllerdetermines that a prefetch limit of cache usage has been met by thecache by establishing that a maximum number of cache blocks in thecongruence class currently contain prefetch values.
 13. The memoryhierarchy of claim 11, said directory entry further comprising anassociated prefetch ID field for storing an identifying number for thecache block.
 14. The memory hierarchy of claim 11, wherein the new valueis an instruction, and the cache is an instruction cache.
 15. The memoryhierarchy of claim 11, wherein the new value is operand data, and thecache is a data cache.
 16. The memory hierarchy of claim 11, whereinsaid congruence class contains a plurality of prefetch values.
 17. Thememory hierarchy of claim 16, wherein said cache controller selects acache block within the congruence class for eviction and said allocationfrom among only those cache blocks containing a prefetch value.
 18. Thememory hierarchy of claim 17, wherein said cache controllerpreferentially selects a cache block whose contents have been sent to anext higher level cache.
 19. A processing system comprising: a processorhaving at least one instruction execution unit; and a memory hierarchyhaving a cache array including a congruence class including a pluralityof cache blocks containing at least one non-prefetch value and at leastone prefetch value; a directory of contents of the cache array, saiddirectory including an entry associated with a cache block containingsaid prefetch value; and cache controller that, responsive to loadingprefetch value, marks the directory entry to indicate that cache blockcontains prefetch value and, responsive to a request for a new valuethat maps to the congruence class, determines by reference to thedirectory that a prefetch limit of cache usage have been met by thecache and, responsive thereto, preferentially allocates the cache blockcontaining the prefetch value to receive the new value.
 20. Theprocessing system of claim 19, wherein said processor includes aprefetch unit that issues the request.
 21. The processing system ofclaim 19, wherein the request is a demand request made in response tothe at least one instruction execution unit processing a demandoperation.